Image & Video Retrieval for National Security Applications: An Approach Based on Multiple Content Codebooks

The views, opinions and/or findings contained in this report are those of The MITRE Corporation and should not be construed as an official Government position, policy, or decision, unless designated by other documentation. Approved for public release; distribution unlimited. Abstract Content-based retrieval of images and video is examined in the context of national security applications (defense, intelligence, and law enforcement). The unique characteristics of CBR systems for these applications are developed and contrasted with those for commercial industry (entertainment, retail, publishing, etc.). The variety of visual media, the lack of internal image structure, the presence of significant raw metadata, and the presence of domain restrictions are discussed, along with other characteristics. A general CBR approach, and its implementation, that is well-suited to the unique characteristics of national security applications is described in detail. This approach is based on the concept of multiple vector quantization codebooks for content extraction, and indexed-based query that uses multiple content types simultaneously.